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DETAILED ACTION 



Response to Arguments 



The Examiner acknowledges the Applicants' amendments to claims 1, 4, 5, 8, 9, and 10, 
and the cancellation of claims 2, 3 , 6, 7, 12, and 13. Regarding independent claims 1 5 4, 8, and 
10, the Applicants allege that neither Klemets (U.S. Patent No. 6,449,653 to Klemets et al.) nor 
Liou (U.S. Patent No. 6,278,446 to Liou et al), as described in the previous Office Action, 
explicitly teach generating a position icon and traversing a hierarchical arrangement of outline 
data in correspondence with the presentation of content data, as has been added to each of these 
claims. In support of this allegation, the Applicants submit that the claimed invention comprises 
a single position icon which moves relative to a hierarchical outline in synchronization with the 
presentation of content data, so that the user may, at any given instant, associate the current 
content data presentation with a dynamically changing location in the outline. The Applicants 
maintain that neither Klemets nor Liou present such an icon, which dynamically traverses a 
hierarchically arranged table of contents in synchronization with the presentation of audio and/or 
video data, and thus the Applicants conclude that the combination of Klemets and Liou fails to 
teach generating a position icon and traversing a hierarchical arrangement of outline data in 
correspondence with the presentation of content data. The Examiner concedes that neither 
Klemets nor Liou present an icon which dynamically traverses a hierarchically arranged table of 
contents in synchronization with the presentation of audio and/or video data. However, and with 
all due respect, the Examiner notes that no claim clearly expresses generating a position icon 
which dynamically traverses a hierarchically arranged table of contents in synchronization with 
the presentation of content data. Rather, each of independent claims 1, 4, 8, and 10 expresses 
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generating a position icon and traversing a hierarchical arrangement of outline data in 
correspondence with the presentation of content data. In other words, although these claims 
recite a position icon and the act of traversing a hierarchical arrangement of outline data, there is 
no recitation that the position icon itself traverses the hierarchical arrangement of outline data. 
The combination of Klemets and Liou, as is more fully described below, teaches generating a 
position icon, and teaches traversing a hierarchical arrangement of outline data in 
correspondence with the presentation of content data. Consequently, and given the broadest, 
most reasonable interpretation of their claim language, Klemets and Liou are considered to 
anticipate claims 1, 4, 8, and 10. 

The Examiner acknowledges the Applicants' amendments to claims 10, 14, and 15, and 
the cancellation of claims 12 and 13. Regarding independent claims 10, 14, and 15, the 
Applicants note that Klemets discloses a client process which accesses all of the audio/video and 
annotations streamed from a file on a server portion of the system, and as such, the data accessed 
by any links are to data within the same data source and the same client/server system. The 
Applicant thus concludes that Klemets does not present linking data linked to a data source 
external to the computer system, as has been added to each of claims 10, 14, and 15. The 
Examiner appreciates the Applicants' attempt, via the submitted amendments, to differentiate the 
claimed invention from that of Klemets; the amendments intended to show that, unlike Klemets, 
the Applicants' invention comprises displaying linking data to data sources external to the 
storage system providing the presentation streams. However, the Examiner respectfully submits 
that claims 10, 14, and 15, even as amended, fail to clearly reflect such a concept, as is evidenced 
by the fact that Klemets is still considered to read upon these claims. As stated by the 



Application/Control Number: 09/764,633 Page 4 

Art Unit: 2173 

Applicants, Klemets discloses a client process which accesses all of the audio/video and 
annotations streamed from a file on a server. This client process is understood to be 
implemented on a first computer system, and the server process is understood to be implemented 
on a second, external computer system (for example, see figure 2, in addition to its description in 
column 2, lines 35-54). Consequently, as is more fully described below, the linking data 
presented on the display of the client computer system is linked not only to audio and video data, 
but also to annotation data on the server computer, and is therefore linked to other data, which is 
supplementary to the presented audio/video content. Additionally, the data source of this 
annotation data, being on the server computer, is external to the client computer system. 

The Examiner acknowledges the Applicants' amendments to claims 16, 17, 19, and 20, 
and the cancellation of claim 21 . Specifically regarding claim 16, the Examiner acknowledges 
the amendment incorporating user selectable options superimposed over a presentation, and 
asserts that Efrat teaches presenting such user selectable options superimposed over a 
presentation, as was described in the previous Office Action, and is again described below. In 
reference to both claims 16 and 20, the Applicants assert that neither Efrat nor Klemets teach 
presenting user selectable options superimposed over a presentation, whereas added to each of 
these claims, the user selectable options are associated with a command capable of accessing a 
data source external to the computer system. The Examiner respectfully disagrees with this 
assertion. Efrat specifically discloses that such user selectable options are superimposed over a 
video and may be selected to access other segments of the video, other videos, or HTML files 
(for example, see column 3, lines 30-67). As applied to the client/server system of Klemets to 
access other segments of a displayed video, the other segments of the video are maintained via 
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an external data source, namely a server, as is described above. Thus with Klemets and Efrat, 
the user selectable options are associated with a command capable of accessing an external data 
source. Assuming, for the sake of argument, that Klemets fails to teach that other video 
segments are maintained via an external data source, it is noted that Efrat specifically discloses 
that the HTML files or video segments, which are accessed in response to the selection of a 
superimposed option, are accessed from an external data source, namely a server (for example, 
see column 18, line 23 - column 19, line 30). 

Claim Objections 

Claim 20 is objected to because the phrase "selected of the user selectable options," as 
recited in the claim, is considered grammatically incorrect. Appropriate correction is required. 

Claim Rejections - 35 USC §112 
The following is a quotation of the second paragraph of 35 U.S.C. 1 12: 

The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the 
subject matter which the applicant regards as his invention. 

Claims 1 and 16 are rejected under 35 U.S.C. 1 12, second paragraph, as being indefinite 
for failing to particularly point out and distinctly claim the subject matter which applicant 
regards as the invention. In claim 1, there is no antecedent basis for "the hierarchical 
arrangement" recited in the claim. Concerning claim 16, there is no antecedent basis for "the 
user selectable options." 
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Claim Rejections - 35 USC §102 



The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 

basis for the rejections under this section made in this Office action: 
A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by another filed 
in the United States before the invention by the applicant for patent or (2) a patent granted on an application for 
patent by another filed in the United States before the invention by the applicant for patent, except that an 
international application filed under the treaty defined in section 35 1(a) shall have the effects for purposes of this 
subsection of an application filed in the United States only if the international application designated the United 
States and was published under Article 21(2) of such treaty in the English language. 

Claims 10, 1 1, 14, and 15 are rejected under 35 U.S.C. 102(e) as being anticipated by 
U.S. Patent No. 6,449,653, which is attributed to Klemets et al. (and hereafter referred to as 
"Klemets"). In general, Klemets teaches a method for delivering audio, video, and annotation 
data from a server to a client over a network. The client computer displays the audio and video 
data, in addition to the annotations, which describe the audio and video data (see abstract). 

Regarding claims 10 and 14, Klemets teaches a method implemented in such a client 
computer system, which has a display and is capable of generating a presentation from a stream 
of data, the method comprising: accessing the stream of data (see column 7, line 59 - column 8, 
line 5); extracting video and/or audio content from the stream of data and presenting this content 
on the display (for example, see column 8, lines 6-9); and extracting outline data representing a 
plurality of data segments within the presentation, whereby the data segments are linked to 
respective segments of the presentation, and whereby the outline data is presented on the display, 
as a table of contents, simultaneously with the presentation of the content data (see column 7, 
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lines 1-45; column 5, lines 39-51; and figure 6). In addition, this outline data also represents at 
least one link to data other than the presented audio/video content associated therewith, the 
outline data being linked to a data source external to the client computer system. For example, 
Klemets discloses that the labels in the table of contents not only link to specific segments of the 
audio/video stream, but also link to data segments in another data stream, specifically an 
"annotation stream" (see column 9, lines 18-36). This annotation stream is maintained and 
provided by an external computer system, namely a server (see column 4, lines 36-54; and 
column 8, lines 26-34). Consequently, Klemets further teaches extracting linking data, namely 
this outline data, which represents at least one link to data other than the presented audio/video 
data associated therewith, the linked data linked to a data source, namely a server, which is 
external to the client computer system. Specifically referring to claim 14, Klemets further 
teaches that this method may be implemented as program code, embodied on a computer- 
readable medium (see column 4, lines 25-35). Such a computer-readable medium having 
program code to implement this method is consequently considered a "computer program 
product," like that recited in claim 14. 

In regard to claim 11, Klemets discloses that the user may pause the display of the 
audio/video content (see column 9, lines 8-17), and upon selection of the linking data in the table 
of contents, establish a link to the annotation data (see column 9, lines 18-36). 

Regarding claim 15, Klemets discloses that the above-described client may be 
implemented by a computer system having a display (see column 3, line 22 - column 4, line 42). 
Such a computer system is considered an apparatus, like that of claim 15, which is for displaying 
content from a data file provided by a server. More specifically, Klemets discloses that this 
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client computer comprises a "video/audio renderer" (see column 7, lines 9-18), which presents 
video and audio content from the data file provided by the server (see column 8, lines 6-9). This 
video/audio renderer is therefore considered a "media engine," like that recited in claim 15. 
Additionally, it is further understood that this client computer, as described by Klemets, 
comprises: program logic for streaming content from the data file and for coordinating a 
presentation of the content by the media engine, the presentation having a plurality of data 
segments, each indexed by a corresponding time stamp (for example, see column 7, line 59 - 
column 8, line 9; and column 9, lines 18-31); program logic for displaying, as an table of 
contents, relevant links from the data stream to a data source, namely a server, which is external 
to the client computer system, i.e. apparatus; (see column 7, lines 9-45; column 5, lines 39-51; 
and column 9, lines 18-37); and program logic for resolving the relevant links to the server (see 
column 7, lines 1-8; and column 9, lines 18-37). 



Claim Rejections - 35 USC §103 
The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

Claims 1, 4, 5, 8, and 9 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
the U.S. Patent of Klemets, which is described above, and also over U.S. Patent No. 6,278,446, 
which is attributed to Liou et al. (and hereafter referred to as "Liou"). Regarding claim 1, 
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Klemets discloses that a client process may be implemented by a computer system having a 
display (see column 3, line 22 - column 4, line 42). Such a computer system is considered an 
apparatus, like that of claim 1, which is for displaying content from a data file provided by a 
server. More specifically, Klemets discloses that this client computer comprises a "video/audio 
renderer" (see column 7, lines 9-18), which presents video and audio content from the data file 
provided by the server (see column 8, lines 6-9). This video/audio renderer is therefore 
considered a "media engine," like that recited in claim 1. Additionally, it is further understood 
that this client computer, as described by Klemets, comprises: program logic for streaming 
content from the data file and for coordinating a presentation of the content by the media engine, 
the presentation having a plurality of data segments, each indexed by a corresponding time stamp 
(for example, see column 7, line 59 - column 8, line 9; and column 9, lines 18-31); program 
logic for displaying an outline, specifically a "table of contents," which is displayed during the 
presentation (see column 7, lines 9-45; column 5, lines 39-51; and figure 6); and program logic 
for accessing one of the plurality of data segments within the presentation upon selection of a 
corresponding portion of the outline of the presentation (see column 7, lines 1-8; and column 9, 
lines 18-36). Thus regarding claim 4, Klemets teaches a method implemented in such a client 
computer system, which has a display and is capable of generating a presentation from a stream 
of data, the method comprising: accessing the stream of data (see column 7, line 59 - column 8, 
line 5); extracting video and/or audio content from the stream of data and presenting this content 
on the display (for example, see column 8, lines 6-9); and extracting outline data representing a 
plurality of data segments within the presentation, whereby the data segments are linked to 
respective segments of the presentation, and whereby the outline data is presented on the display, 
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as a table of contents, simultaneously with the presentation of the content data (see column 7, 
lines 1-45; column 5, lines 39-51; and figure 6). Specifically referring to claim 8, Klemets 
further teaches that this method may be implemented as program code, embodied on a computer- 
readable medium (see column 4, lines 25-35). Such a computer-readable medium having 
program code to implement the above-described method is considered a "computer program 
product," like that recited in claim 8. Regarding claims 1, 4, and 8, as the display of the 
audio/video presentation is directly associated with the user's traversal of the table of contents 
(for example, see column 9, lines 18-37), the user is considered to traverse the table of contents, 
i.e. outline, in correspondence with the presentation of the audio/video content data. However, 
Klemets does not explicitly teach that this table of contents is displayed in a hierarchical 
arrangement, as is expressed in claims 1, 4, and 8, or that a position icon is generated and that the 
hierarchical arrangement of the table of contents is traversed in correspondence with the 
presentation of the audio/video data, as is expressed in each of claims 1, 4, and 8. 

Like Klemets, Liou presents a method for displaying a video, and navigating the video 
via a table of contents. Regarding the claimed invention, Liou discloses that this table of 
contents may be hierarchically organized, whereby each segment in the video is represented by 
an icon in the table of contents, the position of the icon in the table of contents corresponding to 
the position of the segment within the video (for example, see column 4, lines 55-67; and column 
13, line 52 - column 14, line 6). Liou thus teaches displaying a video table of contents, which is 
hierarchically organized, and generating a position icon and traversing the table of contents in 
correspondence with the presentation of the audio/video content. 
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Consequently, it would have been obvious to one of ordinary skill in the art, having the 
teachings of Klemets and Liou before him at the time the invention was made, to modify the 
table of contents taught by Klemets, such that it is hierarchically organized and such that each 
video segment is represented by an icon in the table, as is done by Liou. It would have been 
advantageous to one of ordinary skill to utilize such a combination because a hierarchically 
organized table of contents better depicts the video being presented, allowing for more efficient 
browsing of the video, as do icons representing the segments of the video, as is demonstrated by 
Liou. 

Concerning claims 5 and 9, Klemets discloses that, upon selection of a "content label" in 
the table of contents, i.e. outline, a link associated therewith is resolved and the segment of audio 
and/or video content referenced by the link is extracted from the data stream and presented (see 
column 7, lines 1-8; and column 9, lines 18-36). Consequently, the above-described 
combination of Klemets and Liou is considered to teach a method, like that recited in claim 5. 
As described above, Klemets teaches that such a method may be implemented by program code 
embodied on a computer readable medium. A computer-readable medium having program code 
to implement this method is considered a "computer program product," like that recited in claim 
9. 

Claims 16-20 are rejected under 35 U.S.C. 103(a) as being unpatentable over the U.S. 
Patent of Klemets, which is described above, and also over U.S. Patent No. 6,570,587, which is 
attributed to Efrat et al. (and hereafter referred to as "Efrat"). Concerning claim 16, Klemets 
teaches a method implemented in such a client computer system, which has a display and is 



Application/Control Number: 09/764,633 Page 12 

Art Unit: 2173 

capable of generating a presentation from a stream of data, the method comprising: accessing 
the stream of data (see column 7, line 59 - column 8, line 5); extracting video and/or audio 
content from the stream of data and presenting this content on the display (for example, see 
column 8, lines 6-9); and extracting outline data representing a plurality of data segments within 
the presentation, whereby the data segments are linked to respective segments of the 
presentation, and whereby the outline data is presented on the display, as a table of contents, 
simultaneously with the presentation of the content data (see column 7, lines 1-45; column 5, 
lines 39-51; and figure 6). This outline data is considered selection data, like that recited in 
claim 16, as it represents at least one user-selectable region, i.e. segment, within the presentation 
of the content data, the user selectable region associated with a command capable of accessing a 
data source, namely a server, which is external to the computer, in order to display the region. 
Klemets discloses that, upon selection of such a user-selectable region represented in the table of 
contents, the presentation is modified to display the segment associated with the selection (see 
column 7, lines 1-8; and column 9, lines 18-36). Thus Klemets further teaches: extracting 
selection data representing at least one user-selectable region within the presentation of the 
content data, the user selectable region associated with a command to display the region, 
whereby the command is capable of accessing a data source external to the computer system; and 
modifying the presentation of the content data upon selection of the user-selectable region 
associated with a selectable command. Regarding claim 20, Klemets additionally discloses that 
a single data stream may comprise interleaved video, audio, and annotation data (see column 9, 
lines 37-46). The annotation data comprises event time markers, each marker corresponding to a 
command to retrieve and display annotation data from a specific source (see column 5, lines 52- 
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67; column 6, lines 23-67; and column 8, lines 44-67). Additionally, it is understood that this 
interleaved stream comprises the outline data representing a plurality of data segments within the 
presentation, whereby the data segments are linked to respective segments of the presentation, 
and whereby the outline data is presented on the display, as a table of contents, simultaneously 
with the presentation (see column 7, lines 1-45; column 5, lines 39-51; and figure 6). This 
outline thus comprises user selectable options for navigating to various segments of the 
presentation, the user selectable options capable of accessing a server, i.e. a data source external 
to the client computer system. Klemets further teaches extracting the video and audio 
presentation data from the stream and generating a presentation thereof (for example, see column 
8, lines 6-9), and also, extracting the annotation data, which corresponds to internal commands, 
from the data stream and interpreting the internal commands to retrieve and display annotation 
data (for example, see column 8, lines 44-58). Lastly, it is understood that the user selectable 
options are extracted from the data stream and presented via a table of contents, whereby the 
presentation is manipulated in response to user selection of one of these options (see column 7, 
lines 1-8; and column 9, lines 18-36). Klemets, however, does not explicitly teach 
superimposing such options for manipulating the presentation over the audio/video presentation, 
as is expressed in claims 16 and 20. 

Like Klemets, Efrat discusses the presentation of multimedia data comprising video and 
audio. Regarding the claimed invention, Efrat teaches extracting user-selectable options from a 
video stream, and then superimposing these user-selectable options over the displayed video, the 
user-selectable options selectable to navigate to targets, such as other segments of the video, 
other videos, or HTML files (see column 3, line 30 - column 4, line 3). 
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Therefore, it would have been obvious to one of ordinary skill in the art, having the 
teachings of Klemets and Efrat before him at the time the invention was made, to modify the 
video stream taught by Klemets to include the user-selectable options taught by Efrat, which are 
superimposed over the displayed video. It would have been advantageous to one of ordinary 
skill to utilize such a combination because video may be a desirable means for presenting options 
to the user, as is taught by Efrat (for example, see column 1, lines 15-35). 

As per claim 17, Klemets discloses that the table of contents may display data 
representing a plurality of user-selectable regions within the presentation (for example, see the 
table of contents, reference number 630, in figure 6). Klemets thus teaches extracting data 
representing a plurality of user-selectable regions within the presentation of the content data, and 
whereas described in the previous paragraph, each user selectable region is associated with a 
command. 

Regarding claim 18, Klemets discloses that, upon selection of a user-selectable region 
represented in the table of contents, the presentation is modified to display the segment 
associated with the selection (see column 7, lines 1-8; and column 9, lines 18-36). The user thus 
uses the table of contents to navigate through the presentation of the content data; the user moves 
to a segment of his or her choice in the presentation by using the user-selectable regions 
represented in the table of contents. 

As per claim 19, Klemets discloses that the user may pause the presentation of the 
audio/video content (see column 9, lines 8-17), and upon selection of the Unking data in the table 
of contents, establish a link to another segment of the audio/video data (see column 7, lines 1-8; 
and column 9, lines 18-36). 
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Conclusion 



Applicant's amendment necessitated any new ground(s) of rejection presented in this 
Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). 
Applicant is reminded of the extension of time policy as set forth in 37 CFR 1 .136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1 .136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the date of this 
final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Blaine Basom whose telephone number is (703) 305-7694. The 
examiner can normally be reached on Monday through Friday, from 8:30 am to 5:30 pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, John Cabeca can be reached on (703) 308-3 1 16. The fax phone number for the 
organization where this application or proceeding is assigned is 703-872-9306. 
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Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 



btb 




^ JOHN CABECA 
SUPERVISORY PATENT EXAMINE^ 
TECHNOLOGY CENTER 2100 



